\documentclass[12pt]{article}

\usepackage{fullpage}
%\usepackage{url}

% Titlepage
\title{Robot-augmented safety in machine shops\\}
\author{
	Aaron Fineman\\
	\small{Robotics Engineering}\\
	\small{Worcester Polytechnic Institute}\\
	\small{Worcester, Massachusetts 01609}\\
	\small{aaron@wpi.edu}
	\and
	Prof. Sonia Chernova\\
	\small{Computer Science Department}\\
	\small{Robotics Engineering Department}\\
	\small{Worcester Polytechnic Institute}\\
	\small{Worcester, Massachusetts 01609}\\
	\small{soniac@wpi.edu}
}
\date{2 March 2011\\\small{Prepared in partial fulfillment of the requirements of the\\ special topics course on Human-Robot Interaction}}

\begin{document}
\maketitle
\newpage
%\tableofcontents

\abstract
A significant number of people are injured or permanently maimed due to accidents in machine shops. While tools have gotten safer, they have reached a plateau and we believe an external source is required to further safen machine shops. This project makes use of a video camera and an industrial arm to intercept a operator's arm if it is about interact unsafely with a machine. However, there is still a lot of work to be done in both computer vision and robotic manipulation before this is truly viable.

\section{Introduction}
A significant number of people are injured or maimed due to accidents in machine shops, and many deaths still occur. Brookhaven National Laboratory reports 18,000 annual amputations, lacerations, crushing injuries, abrasions, and over 800 deaths\cite{BNL:safety}. Common causes of injury in machine shops are, but not limited too, coming into contact with the machine, or being trapped between material and the machine, becoming entangled in the machine, being struck by ejected machinery parts, or being struck by ejected material\cite{HKUST:safety}.

There is very little work being done on external methods to prevent injury in machine shops, most research is focused on building more advanced safety features into the machines themselves.

The most popular method is capacitive sensors to detect flesh, and stop the blade as soon as possible (potentially destructively.) SawStop is a leading company in this area. They market a system that can stop a saw blade from rotating at 4,000 RPM in less than 5ms (destructive to the blade only), which they claim will only cause a nick on ones finger\cite{sawstop}. However 4,000 RPM is only sufficient for cutting wood, metals are cut at a higher speed, which may result in a longer braking time, which they do not reference.

In addition to brakes, some machines implement guards. Most guards are reactive, to prevent further injury, such as limbs being pulled into the machine, however, there are some proactive guards, such as two-handed control devices, which only function if both hands are on the controls\cite{HKUST:safety}.

But with all the technology available, a lot of the safety implemented is sociological. In many shops, including the WPI ones, there are simple rules covering proper eye protection, and loose clothing restrictions to avoid becoming caught or pulled into machines\cite{WPI:safety}.

\section{Approach}
We decided to experiment with an external safety system that would augment the existing ones. Our vision was a video camera attached to each machine aimed at the workspace and tool. Next to the machine would be a mounted industrial arm, whose workspace coincided with that of the tool, and included unrestricted access to the operator's limbs. If the operator was in danger of intersecting with the tool, the industrial arm would be able to shield the tool, or in worse case, grab the operator's arm and remove it from harm.

%This project spans two courses, WPI-CS/RBE 595 Human-Robot Interaction and WPI-RBE 501 Robot Dynamics. In it's current state, this project mainly reflects Human-Robot Interaction, at the time of writing, Robot Dynamics is still in progress.

For the industrial arm, a FANUC LR Mate 200iB six-axis arm was used due to availability\cite{fanuc}. The available gripper has no force feedback, a separate module is necessary. 

Software was developed under a WMware Fusion 3.1.2 image of Xubuntu 10.10 utilizing OpenCV 2.1.0. The video camera used was the built in camera on a Macbook (640x480 output). The code was written in python, and is available freely under the BSD license\cite{google-code}. To capture a frame of video from the camera, v4lctl (provided by xawtv) was invoked via a system call, which then transcoded and wrote it's output to a temporary file, which was then loaded back into OpenCV. The incoming images were then converted to grayscale and passed through a haar classifier cascade. Due to available time, an in-house haar classifier cascade was not trained, but a freely available, open-source one was made use of\cite{haar-hands}.

\section{Results}
Overall, there was significant lag capturing frames from the camera's feed. This was determined via wallclock to average around 2 seconds. When moving at approximately 1ft/s at a distance of 2 feet, the hand was able to traverse the viewport (1ft/s being an acceptable speed for cutting wood with a circular saw\cite{sawstop}.)

The haar classifier cascade that was used had trouble detecting hands that were not spread out (at least one clear verticle finger), and was not able to detect hands at an angle. Multiple hands were detected fine, however multiple hands led to an increase in mistaking other patches of skin for hands. Background clutter that was not skin-tone did not seem to interfere with hand detection, as long as the hand maintained the proper angles and clear fingers. Held objects introduced some error, but hands were still detected as long as fingers were visible. However, this required holding objects in a very unnatural position, and had limited range (the maximum range for this pose seemed to be under 4 feet.)

The haar classifier cascade was surprisingly resilient to skin tone. Under proper lighting, a variety of skin tones ranging from black to white were detected flawlessly. However, under suboptimal lighting, only white and light brown were able to be detected.

Sample videos were submitted with this paper, and are also available in the code repository\cite{google-code}.

\section{Discussion}
A significant problem with the system in it's current incarnation is the lag in the video camera. While there was not time to compare the latency of the python script to a natively compiled C++ version, any speed differences were eclipsed by capturing frames from the video stream. It appears that the use of an external system call, and using a file as a pipe introduced a larger speed trade-off than anticipated. While a C++ rewrite may not be necessary, this section of code should be replaced by one that initializes the camera, and simply captures a frame from the stream when required.

We were also largely unsatisfied with the haar classifier cascade that we used. If it were not for time constraints, we would have set up a video camera in the machine shop, as our system would only be working under very constrained environments. The poses the haar classifier cascade was trained on were largely spread hands, and seemed to rely on fingers for detection. This only allowed for straight-on views of the hand, any large angle obscured the fingers, and caused the haar classifier cascade to fail. Common positions for feeding material through saws involve a slightly curled hand, which would not currently be picked up.

Other minor software problems encountered involved the boxes around hands at close ranges (approximately 2 feet) were excessively large, and prone to error and jitter. They were mostly accurate at 4-6 feet, but still prone to jitter and losing the hand. This does not suggest that they would be able to reliably track fingers in a typical machine shop environment.

\section{Conclusion}
In conclusion, we do not believe that vision systems are in a position to reliably track hands in a machine shop environment combined with an external safety system.

For this system to even stand a chance of being competitive with capacitive braking systems (approximately 5ms response time,) a complete rewrite of this system would be required, making use of a video stream rather than discrete photo captures, and a natively compiled language. In addition, haar classifier cascades may be less appropriate than a well-tuned edge identification algorithm (machine shops are a largely controlled environment.)

This research suggests that for now, better built in safety devices on machines and good shop guidelines will be able to reduce accidents the most.

% References
\addcontentsline{toc}{section}{References}
\bibliographystyle{ieeetr}
\bibliography{references.bib}

\end{document}

